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ABSTRACT 


Using  configural  polysampling**  to  identify 
high-performance  estimates  proceeds  in  two  phases: 

*  estimating  the  best  compromise  to  make 
at  each  configuration  polysampled, 

*  choosing  a  generally-defined  estimate 
whose  performance  comes  close  to  that  of 
this  estimated  optimum. 

In  the  first  step,  shadow  pricing  of  the  criteria 
for  the  different  situations  involved  is  helpful. 

If  the  criteria  are  the  relative  excess  vari¬ 
ances  for  the  situations  in  question,  minimizing 
their  maximum  corresponds  to  maximizing  polyeffi¬ 
ciency.  In  this  case,  as  in  most  others,  finding 
the  shadow  prices  is  likely  to  require  iteration. 

In  the  second  step,  whether  we  use  a  regres¬ 
sion  or  selection-by-sectors  approach,  or  combine 
both,  the  same  shadow  prices  are  useful. 

This  report  sketches  some  of  the  approaches 
to  the  two  steps  that  seem  natural . 

**See  Technical  Report  No.  185  by  Pregibon  and 
Tukey  for  the  ideas  of  configural  polysampling. 

See  Technical  Report  No.  191  by  Bell  and  Pregibon 
for  an  implementation. 

i 


1.  Introduction  to  A  and  B 


One  of  the  purposes  of  configural  polysampling  is  to 
identify  —  on  the  configurations  sampled,  at  least  --  esti¬ 
mates  of  high  polypcr formance,  and,  as  well,  to  assess  the 
extreme  possibilities  of  specific  kinds  of  polyperformence. 
Often  these  polyperformances  are  functions  of  the  various 
sampling  variances,  functions  of  such  a  character  that  the 
family  of  Sayeb  estimates  --  of  estimates  minimizing 

S  [(shadow  price  for  (Q)  *  (va^ce(  ^  Q)_: _ (constant)! 

L  (constant*)  J 


for  some  set  of  shadow  prices  --  includes,  by  necessity,  the 
estimate  with  highest  polyperformance. 


Below,  we  first  discuss,  in  some  generality,  the 
evaluation  of  Sayeb  estimates  in  terms  of  shadow  prices  and 
then,  second,  a  plausible  approach  to  iterative  selection  of 


the  shadow  prices  in  the  special  case  where  polyefficiency 
is  the  polyperformance  to  be  maximized.  In  this  case,  of 
course,  we  can  equally  well  treat  the  largest  relative 


excess  variance 


max 

0 


(variance  at  Q)  -  (minimum  variance  at  Q) 
minimum  variance  at  Q 


as  a  polyperformance  to  be  minimized . 


We  shall  work  with  configurations  specified  as 
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c, ,c_,...,c  ,  where  c  =0,  c.=l  and 
l  i  n  at) 

y.  =  r  +  c.s 

where  y^  <  <  • • •£  yn  are  the  order  statistics  of  a  sam¬ 

ple,  and  a  and  b  are  likely  to  be  chosen  close  to  n/4  and 
3n/4.  (Note  that  r  and  s  are  coordinates  locating  a  partic¬ 
ular  sample  in  the  configuration;  they  do  not  have  their 
most  customary  statistical  meanings.) 

A 

The  Sayeb  mi nimi zers 
2.  The  case  of  two  situations. 

We  start  with  only  two  situations,  A  and  Z,  and  any  one 

configuration.  If  t  locates  the  estimate  relative  to  the 

configuration,  as  when  T=r+ts,  the  conditional  mean  square  error 
of  the  estimate,  for 

/this  configuration,  is  a  quadratic  function  of  t.  We  lose 
nothing  if  we  modify  the  scale  of  t  so  that  t  =  0  is  the 
optimum  value  of  t  in  situation  A  and  t  =  1  is  optimum  for 
situation  Z. 

If  T^a  and  T £2  were  t^ie  original  optimum  values,  any 
value  of  t  would  correspond  (where  r  and  s  locate  a  sample 
in  the  configuration  f)  to 
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f 


Var(r+(TfA 


+  t(TfZ-TfA))s) 


=  (minimized  A-variance) ^  +  t' 


(T£Z-T£f.»24. 


and  thus  to  a  contribution  to  the  weighted  sum  of  excess 
variances  at  situation  A  equal  to 


t2"-Ms£A(T£Z-TfA)2> 


where  V,’  is  the  weight  on  configuration  f  appropriate 
for  situation  A.  (In  dealing  with  operators  like  "var" 
which  may  involve  a  restricted  range,  we  specify  the  range 
above  the  operator,  reserving  the  space  below  the  operator 
for  binary  variables.  Thus  the  "  f  "  above  the  "var"  above 

I 

indicates  a  conditional  variance  over  the  configuration  f.) 
(The  "weights"  used  are  discussed  in  Technical  Report  #185.) 

If  we  are  concerned  with  polyefficiency,  we  are  con¬ 
cerned  with  the  excess- var i ance  as  a  fraction  of  the  minimum 
variance  (minimum  when  A  alone  is  considered)  which  we  cell 
the  "minimum  A-variance".  Thus  our  concern,  at  f  under  A, 
about  t  differing  from  zero  is  the  shadow  price  times 


KAfSfA(Tf  Z^fA5 


minimum  A-veriance 


'  aAf C 


while  that  at  f  under  Z  is 
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yZfSfZ  (Tf7~TfA)  =  H-t)2 

minimum  Z-variance  ~  aZf  ' 

If  the  shadow  prices  are  c(  and  f;,  we  then  wish  to 
minimize 

^  *  aAffcf  +  ^  ^  aZf^1_tf^ 
f  f 

Differentiation  w.r.t.  t  ^  leads  to 

<<SAftf  "  ^aZf(1'tf)  "  ° 

and  thence  to 


and  hence  to  a  total  price  that  is  the  sum  over  f  of 


<«aA£)  «jaz£) 


More  interesting,  usually,  will  be  the  individual 
excess  variances 
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and 


If 


and 


which  ■ 
1/25  for 


-  5  - 


2  aAFfcf 


I  a 


*2a2 


Af 


Zf 


(c*a 


Af 


+  $aZf} 


$25 


JZf 


aAfaZf 


(<<«»*  +  ^azf) 


‘Af 


S  a2f(l-tf)  -  S  azf 


H2*2 
*  aAf 


WaA£  *  ^Zf>2 


d2S  a 


aAfaZf 


A£  (“*A  £  +  Hf)2 


we  introduce 


rf 


Af 

Zf 


*  r~+~T(T7r)  *  sech2(ln  r) 

1  if  r  -  1,  about  1/3  for  r  -  0.1,  or  10,  and  about 
r  ■  .01  or  100,  these  the  excess  variances  become 
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-  6  - 


5  aftft2  =  (1/4 )  (^/«)  2  a2f*J(rf) 


2  azf(l-tf)2  =  <l/4)M/$)  2  aA£rf(rf) 


which  will  be  less  than,  but  perhaps  of  the  order  of 

"4  S  azf 


4^  5  aAf 


What  if  there  are  more  than  two  situations? 


3.  Three  situations. 


Suppose  now  that  we  consider  an  additional  situation, 
B,  one  where  the  relative  excess  variance  is  estimated  by 


S  »B£(t-b) 


where  t  *  b  ( not  necessarily  between  0  and  1)  is  the  optim¬ 
izing  value  of  t  for  situation  B  alone.  With  shadow  prices 
c(,p  and  we  are  to  minimize 

2  «a.f t2  +  pa  f(t-b)2  +  $a  f(t-l)2) 


We  can  simplify  this  by  using 
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f 


(l-c)t2  +  c(l-t) 2  -  c(l-c)  =  (t-c)2 

an  identity  easily  checked.  Changing  notation,  we  have 

paBf(t-b)2  =  PaBf  (1-b)  t2  +  paBfb(t-l)2  -  paBfb(l-b) 

and  what  we  are  to  minimize  becomes 

S  |  (daAf-paBf  (1-b)  )t2  +  ($azf+{3aBfb)  (t-1)2  -  5  paBfb(l-b)  j 

where  the  final  term  is  independent  of  t. 

By  analogy  with  the  last  section,  we  see  that 

.  **Af  +  PbaBf 
f  daAf  +  paBf  +  ^azf 

and  we  may  as  well  calculate  the  three  excess  variances  as 

EA  *  S  aAf'f 

“B  '  S  •Bf<tf-b)2 

Ez  ' 1  azf(tr1)2 

respectively. 

4-.  Fouc  or  more  situations. 

The  formulas  for  four  situations  follow  in  an  entirely 
similar  way.  From  them  the  general  relations  lead  to  the 
Sayeb  minimizations  (Eft,  E0 ,  E^, , . . .  ,EZ>  given  by 
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max { E^, Eg  » i • • • » ^ 

and  that  we  have  tried  4  (0  }  ,p  (0  (0  )  with  results 

E_(0),  E_(0)  E_  (0 )  ,  whose  maximum  is  E„a„(0). 

D  L  luoX 

If  E. (0)  is  less  than  E  (0),  we  are  doing  wastefully 
a  max 

well  at  A,  so  that  we  want  to  reduce  the  corresponding 
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relative  shadow  price,  c(,  and  let  Eft  increase  in  the  hope 
that  E  decreases.  Similarly  if  ED(0)  is  less  than 

m  a  X  d 

E  ,  (0),  we  would  like  6  to  sink  relatively.  And  so  on. 
max  r 

Let  us  propose  one  algorithm  that  moves  things  in  the  right 
d  irection . 


For  some  exponent  p  --  we  try  p  =  1  (or,  perhaps, 
p  =  0)  for  awhile,  and  then  learn  to  choose  p  more  reason¬ 
ably  —  and  some  convenient  k,  we  can  take 


I  E.  (0 )  |p 

a(1>  *  "'irSor'  a(0> 

I  max  I 


I  E(0)  lP 

P(1)  *  k|E-°~  (0)1  P(0) 

rmax(U)  I 


!  e  _  ( o )  ! p 


<(1’  *  k&r| «'0’ 


(It  might  be  nice  to  choose  k  so  that 

c(  (1  )+p  (1 )  +  . . .  +{•  (1 )  =1000  —  or  some  other  handy  number.) 


Looking  at  a  few  successive  values  of 
{  (c(  ( i)  ,p  (i)  , . .  ,4  ( i)  )  }  might  now  suggest,  for  instance,  try¬ 
ing  certain  of  these  shadow  prices  as  zero.  If,  while  using 

them  as  zero,  the  corresponding  E  ever  becomes  E  ,  we 

m  sx 
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should  make  them  finite  (>0)  again. 

This  may  not  be  the  most  rapidly  convergent  algorithm, 
but  we  can  hope  for  relatively  good  performance,  once  we 
tune  p. 

C 

External  description  of  estimates 
6_.  Introduction  to  C. 

We  have  discussed  how,  in  simple  instances  we  can  esti¬ 
mate  internal  descriptions  of  "best"  estimates,  finding  out 
—  for  a  sample  of  configurations  —  good  estimates  of  what 
the  value  of  the  "best"  estimate  is  for  that  configuration. 
(The  failure  of  our  "estimates"  to  be  perfect  comes  from  our 
slight  imprecision  in  finding  the  right  shadow  prices.) 

We  now  want  to  estimate  an  external  description  of  the 
"best"  estimate.  This  means  that  we  need  to  find  a  func¬ 
tion,  defined  for  all  configurations  that  comes  close  to  our 
estimate  values  for  each  of  our  sample  configurations. 
Several  approaches  are  possible  including: 

1)  Selecting  a  set  of  explicit  estimates,  evaluating 
each  at  each  selected  configuration,  and  doing  a 
linear  regression  of  our  "best"  values  on  the 
estimate  values  —  the  only  question  is  what 
weights  to  use  in  the  regression. 


11 


2)  dividing  the  space  of  configurations  up  into 
parts,  which  we  will  call  sectors,  evaluating  the 
performance  of  each  of  the  explicit  estimates  for 
each  sector  (using  excess  variances  and  the  optim¬ 
izing  shadow  prices),  and  selecting  the  explicit 
estimate  that  performs  best  in  each  sector, 

3)  continuing  (2)  by  trying  to  interpolate  sensibly 
between  the  several  explicit  estimates  thus 
selected,  one  for  each  sector, 

4)  combining  (1)  and  (2),  which  leads,  after  help 
from  experts,  to  a  solvable  linear  programming 
problem,  and  to  different  lincoms  in  different  sectors 

5)  taking  a  relatively  good  explicit  estimate,  iden¬ 
tifying  (in  terms  of  excess  variance  at  optimizing 
shadow  prices)  its  unnecessary  loss  at  each  confi¬ 
guration,  isolating  the  large-loss  configurations, 
and  studying  the  change  in  estimate  Required  at 
each  of  them  as  a  basis  for  inventing  possibly 
improved  estimates. 

We  will  have  to  learn  by  experience  which  of  these  seem 
most  effective.  A  little  consideration  will  clear  up  cer¬ 
tain  aspects  of  what  we  might  do. 

7.  The  regression  approach . 

Suppose,  then,  that  we  have,  for  s  sample  of  configura- 
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tions  f,  and  two  situations,  A  and  Z: 

$  The  overall  shadow  prices  d  and  $  that  minimize 
the  maximum  relative  excess  variance. 

*  for  each  f,  the  constants  az^  and  t^  (the 

latter  on  the  modified  f-scale  where  t=0  and  t=l 
are  the  separate  optina  and  do  not  correspond  to 
ya  and  yb) 

<>  for  each  of  m  explicit  estimates  the  values  t^. 
for  the  i-th  estimate  of  t. 

Take  first  m  =  2.  We  want  to  consider 

©(estimate  1)  4  (1-0)  (estimate  2) 

whose  t-value  at  f  is 

et£1  4  <l-e>t£2 

The  shadow-priced  cost  of  this  estimate  is 
d  I  eAf (etfj+d-eitf^)2  4  5  s  azf d-etfl- (i-©)tf2)2 

which  equals 

c(92  5  aAft2x  4  2«9(l-e)  S  aAftfl  4  c((l-©)2  5  aAft22 
4  $e2  5  azf(l-tf2)2  +  2$9(l-9)  Szzf  (l-tfl)  (l-tf2) 
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+  $(l-9)2  5  azf(l-tf2) 


(<92  [All]  +  2(<9  (1-9)  f A 1 2 ]  +  c*  (1-9)  2  [A22] 


+  $92[Z11]  +  2$9(l-9)  tZ  1 2]  +  ^  (1-9)  2  fZ  22] 

where  the  six[  ]'s  designate  the  six  sums  written  down 
above.  If  we  write 


' 


[11]  *  ^ [All]  +  ^  tZ 11 ] 

[12]  *  «[A12]  +  $[Z12] 

[22]  *  d  [A22  ]  +  $[Z22] 

where  the  first  and  last  are  the  prices  of  the  indivi¬ 
dual  estimates,  the  price  of  the  linear  combination  is 


92  [11]  +  29  (1-9)  [12]  +  ( 1  —9 ) 2  [22] 


whose  9  derivative  is 


29  [1 1  ]  +  2  (1-29)  [12]-2  (1-9)  [22] 


which  vanishes  when 
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&r 


[22]  -  [12] 
til]  -2 [12]+  [22] 


1 


[111  -  [12] 

[11]  -  2 [12]  +  [22] 


and  the  price  of  the  optimum  linear  combination  reduces 
to 


([11]  [22])-[12])2([ll]  +  [22])  -  2[12]  [11]  [22]  +  2  [12] 3 


divided  by  the  square  of 


[11]  -  2 [1 2 ]  +  [22] 


extending  this  numerically  to  larger  m  offers  little 
difficulty,  since  what  we  are  doing  is  equivalent  to 
seeking  a  minimum  "variance"  when 


[Zii]  -  5  aZf(1"tfi)2 
[Zij]  -  5  azfU-tfi)  <l-tfi) 

The  extension  to  two  estimates  and  more  than  two  situa¬ 
tions  proceeds  in  a  similar  way.  There  will  be  more  shadow 
prices  and  more  sums,  but,  after  pricing  out,  the  same 
number  of  "variances"  and  "covariances". 
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£.  Sector  techniques . 

If  we  were  working  with  situations 
A  *  Gaussian 

2  *  Slash 


we  would  probably  want  to  define  sectors  in  terms  of  some 
measures  of  apparent  Gaussiantity-vs.-stretchedness,  such  as 


K 

K 

K 

K 


1 

2 

3 

4 


javaj  In  s' 


2  2 
In  s^.  -  In  sz 


Cn  "  C1 


Cn-1  *  C2 


where  "javaj"  stands  for  " jackkni fe-estimate  of  variance  of 

2 

the  jackknifed  values  of",  s^.  is  the  robust  estimate  of 
variability  described,  inter  al i a ,  in  Chapter  11  of  Mostell- 
er  and  Tukey,  and  { ci »c2' *  *  * ,cn^ '  are  order  statistics  of 
the  configuration  in  question. 


We  might  want  to  use  such  measures  either  alone  or  in 
combination  with  some  measures  of  skewness,  such  as 


*  (abmean  of  c^’s)  -  (mean  of  c^’s) 

Lj  *  (abmean  of  c^’s)  -  (median  of  c^'s) 

L3  ’  cn  +  C1 
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L4  *  cn-l  +  c2 

where  the  abmean  is  the  mean  of  the  -  central  order 

statistics  (out  of  n) . 

Having  chosen  sectors,  we  now  need  to  estimate  the  per¬ 
formance  of  each  external  estimate  at  the  configurations  in 
our  polysample  that  fall  in  each  sector.  When  we  need  to 
look,  for  each  estimate  and  sector: 

*  for  each  configuration,  at  the  contributions  to 

2 

relative  excess  variance,  say,  aAffcfi  ancJ 
2 

az^(l-tj.)  )  (recall  that  the  a's  incorporate  the 
relevant  weights)  , 

*  for  each  sector,  at  the  priced-out  contributions 

«  S‘  aAf Cf i  *  *  S‘ 

where  the  sums  are  over  all  configurations  in  the 
sector  from  our  polysample . 

If  we  are  selecting  estimates,  we  need  now  only  select 
for  each  sector  the  estimate  whose  sector  price  is  least. 
(We  may  be  able  to  make  good  use  of  a  listing  of  excesses  of 
these  estimated  sector  prices  over  the  similar  estimate  for 
our  optimum  as  defined  by  the  t^.) 

To  go  a  useful  step  further,  we  might  list  the  esti¬ 
mates  in  order  of  increasing  sector  prices  for  that  sector, 
and  then  try,  in  each  sector,  all  50-50  combinations  of  the 
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best  10  (or vbest  6)  estimates  so  far  tried  for  that  sector. 
Sufficient  iteration  here  should  lead  to  close  to  the  op¬ 
timum  linear  combination  for  each  sector. 

A  regression  calculation  in  each  sector  would  also  be 
feasible. 

Everything  is  easy,  so  long  as  we  are  prepared  to  stick 
to  the  previously  found  shadow  prices.  We  will  only  get 
into  linear  programming  situations  etc.  if  we  insist  on  us¬ 
ing  more  correct  shadow  prices  —  more  correct  because  we 
are  no  longer  accommodating  as  wide  a  variety  of  possible 
choices,  thus  shrinking  the  attainable  choice  set  and  tilt¬ 
ing  the  tangent  hyperplane  at  the  new  optimum  This 
corresponds  to  insisting  on  controlling,  for  our  external 
estimates,  the  actual 

maximum  relative  excess  variances 

for  the  more  restricted  situation,  instead  of  controlling 
that  particular 

lincom  relative  excess  variances 

whose  unrestricted  optimum  over  our  sampled  configurations 
enforces  an  optimum  for  the  maximum.  Those  who  wish  to 
modify  may  do  so. 

£.  Use  of  judgment ,  intuition,  insight  and  inventiveness 

Approaches  (3)  and  (5)  above  call  for  human  input.  We 
will  have  to  learn  how  to  do  this  by  trying. 
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